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METHODOLOGICAL CONSIDERATIONS IN EVALUATIVE RESEARCH 
INVOLVING DISADVANTAGED POPULATIONS 



America's War on Poverty has produced, and can be expected to continue 
to produce, various strategies designed to improve the circumstances of disadvan- 
taged persons. The evaluation of the effectiveness of these strategies constitutes 
an important and challenging research problem. Because research with disadvantaged 
groups is still relatively new, dependable guidelines for conducting such research 
are not yet available. During the last two years, we have been conducting studies 
In this area and have found It necessary to experiment with new techniques. This 
paper at tenets a critical analysis of our experience in the hope that the methodo- 
logical considerations developed in the course of our research may be of value to 
other research efforts. 

Selected urban out-of-school Neighborhood Youth Corps programs in five 
cities have been studied during the last two years. Our research design reflects 
the difficulty in conducting research with these populations. 

The complexity of the milieu in which anti-poverty programs operate 
makes it extremely difficult to identify causal relationships betwecm program 
effort and program effect and requires the use of control groups in order to have 
a baseline for judging program effects. The long-range objective of youth pro- 
grams, to help the youth become functioning adults, makes it essential that the 
studies be longitudinal and cover a time span of sufficient length to permit a 
determination as to the adult adjustment of the youth. The design of our studies, 
therefore, included experimental and control groups to be followed through inter- 
views for a five-year period. 



In the course of our research, we have learned some of the hard lessons 
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of implttnenting a longitudinal* experimental-control group design* with. the nega- 
tive lessons of what not to do often as Important as the positive Indications of 

which researdi methods are essential to productive research and can be carried 

* 

through effectively. These conclusions are presented here in the belief that they 
may serve researchers and program personnel interested in the evaluation of the 
effectiveness of programs operating in areas of socio-economic disadvantage. 
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Selection of Study Groups 



Experlmental~control group design logically begiiis With the selection 
of the Individuals vho will comprise the experimental group— the sample of the 
population which has undergone the experience whose effects are being examined; 
and, In our study of MG programs, we constituted experimental groups of enrollees 
and ex'-enrollees In several cities* In general, the size of the sample deter- 
mines the number of variables which can be analyzed* In the NYC research, .the 
many variables of Interest made relatively large samples necessary, and experi- 
mental groups of from 125 to 150 Individuals in each city were constituted in 
order to assure completed interviews for at least 100 respondents* 

In various parts or phases of our research, we used three different 
methods for selecting experimental study groups: (a) groups composed of all 

Individuals entering the program within a selected period of time (Entry Sample) ; 
(b) groups composed of individuals terminating from the program within a selected 
period of time (Termination Sample); and (c) samples selected at random from all 
individuals enrolled In the program on a selected date (Current Enrollee Sample) • 
Broadly speaking, since all experimental study groups represent the program 
population of research interest, they could be termed "samples •’* As we shall see, 
the characteristics of the program population reflected In the experimental study 
group vary, to some extent, with the selection techniques employed. Strictly 
speaking, therefore, the designation "sanq>le" should be reserved, perhaps, for 
those study groups that are constituted by conventional randomized sampling pro- 
cedures* 

Each of these methods possesses practical and theoretical advantages 
and disadvantages which should be recognized by the researcher* The different 



methods of sampling do not provide equivalent samples. In general, the current 
enrollee sample is more likely to include the long-tenure subject and will over- 
estimate the average time an individual is likely to stay in the program. Over 
a sufficiently long period of time, the entry and termination samples will be 
equivalents but, over short periods of time, the termination sample will reflect 
the rate of entry into the program immediately prior to the selection of the 
sample. Thus, after periods of high intake, the termination san^le is more likely 
to include short-tenure subjects, and after periods of low int^e, it will be 
more likely to Include long-tenure subjects. The time span that should be allowed 
for the development of post-program data obviously differs with the selection 
method used. The termination sample, for example, minimises the chronological 
difficulties of follow-up information. In a particular research project, the 
implications of the various methods of study group selection in terms of group 
characteristics and data availability should be carefully weighed. The characteris- 
tics of the three types of samples are discussed in more detail in the following 

sections. 

Entry Sample 

Selecting the experimental study group at the time the subjects enter 
the program has the important advantage of assuring equal exposure to the program 
components. The optimum time span during which the sample is selected is deter- 
mined, in part, by the effect of opposing tendencies. A short period of time 
reduces the effect of program changes but Increases the effect of seasonal varia- 
tions in the composition of the sample. For example, tiie applicants applying for 
the program in Septeiher, December, and June may differ significantly due to such 
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factors of school enrollment, and employment conditions. Seasonal variations 
I are, of course, reduced by selecting the sample at random over the span of a year* 

A period of sample selection of this length, however, increases the effect of pro*- 
gram changes. During the course of a year, the program is likely to be changed 
significantly and the subjects entering at the beginning of the year are exposed 
to markedly different program Influences than those who enter at the end of a 
year. The researcher must analyze each situation he plans to research and develop 

c 

the best compromise between reducing sample or program fluctuations. 

From a practical point of view, the entry sample also suffers from 
length of time necessary from the date the sample is selected until information 
about program effects is available. In general, in studies of out-of-school NYC 
programs, it takes between three and six months to constitute samples of the re-^ 
qulred size for a sophisticated study, the subjects can stay in the program as 
long as two years, and then an additional period of at least six months is needed 
for the subject to acquire sufficient post-NYC experience to permit an evaluation 
of program effects. A time span of over three years between the beginning of 
the 8tu(fy until useful information is available for all the subjects is often 
too long for practical evaluation purposes. Program administrators usually need 
the Information at an earlier time if it is to be useful to them for improving 
programs* Another adverse result stemming from the long time span between selec- 
tion of the sample and information about outcomes is the possibility that policy 
change uay outdate the information, and, thus, the research reflecting an earlier 
I period of the program has become obsolete. 

Retrospective studies, in which the sample is selected from an earlier 
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time period, can reduce the time lag between the Initiation of the research and 
the report of the findings* Such retrospective entry samples necessarily involve 
the use of program records, and the adequacy of a program’s files to provide 
selection and program data must be considered by the researcher* Even if program 
records are adequate, the possibility that the information may be outdated because 
of policy changes must also be taken into account. 

We have concluded from the above considerations that entry samples are 
useful for intensive studies of program components and policies, and are most 
likely to develop theoretical insights into program functioning* On the other 
hand, because of the long time lag between the initiation of the study and the 
availability of results, they are of only limited value for evaluating the effec- 
tiveness of particular programs and suggesting corrective action. 

Current Enrollee Samples 

A sample taken at random from enrollees participating in the program 
at any given date has the advantage of reducing the time between the constitution 
of the sample and the completion of the research. It also simplifies the sample 
selection process since the exact composition of the population from \diich the 
sample is to be selected is known at the time of sample selection, which is not 
always true for an entry sample. For example, after having committed himself to 
an entry sample, the researcher may find that a budgetary freeze has virtually 
stopped enrollment. He cannot then change his design but must, of necessity, 
wait out the freeze, and in the meantime, some of his experimental subjects may 
have terminated from the program while others have not entered. 

The disadvantage of the current enrollee method of sample selection is 
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that It tends to overestimate the average stay in the program since long-term 
enrollees have a better chance of appearing In the sample than short-term en- 
roilees. Another disadvantage of this method is that all data or program ex- 
perience of subjects prior to the date of sample selection must be collected 
from records* 

Termination Sample 

Samples selected from all enrollees terminating from the program within 
a prescribed period of time permit the shortest time lag between program ex- 
perience and follow-up information. These samples are also relatively easy to 
constitute and, on any selected follow-up date, provide for approximately equal 
periods of post-NYC experience for each subject in the sample. 

There are, however, almost always distortions in the composition of 
termination samples related to the rate at which new enrollees have been enter- 
ing the program. It was found, for example, that a termination sample selected 
in a period directly following intensive recruitment tends to Include more sub- 
jects with short tenures; whereas, a sample selected at the end of a period of 
restricted recruitment tends to have a higher proportion of enrollees with long 
enrollment. Thus, any termination samples selected during the first two years 
of the NYC program must Inevitably underestimate the average length of time en- 
rollees stay in the program because there will be some enrollees who have not 
yet terminated from the program. Table 1 demonstrates this effect by comparing 
a termination and a current enrollee sample selected from the same program and 
approximate time period. 






TABLE 1 



COMPARISON OF TERMINATION AND CURRENT ENROLLEE SAMPLES 
ON LENGTH OF TOUR IN N.Y.C. 



Months in Program 


Current Enrollee 
Sample 


Termination 

Sample 




0-3 months 


13% 


30% 




4-9 months 


35% 


36% 




10 months or more 


52% 


34% 






TOTAL 100% 


100% 




Average number of months 


10.3 


6.0 





The current enrollee sample averaged significantly longer tenure, ten 
months after the selection of the sample, than did the teminatlon sample* About 
a quarter of the current enrollee sample was still enrolled In the NYC at the 
time the study was completed. When sufficient time has elapsed for all these 
subjects to have completed their tour, the difference between the two samples 
will be even greater. 

It is thus concluded that termination samplers are useful for providing 
relatively quick feedback on program effects, but that the samples arc highly 
sensitive to changes in recruitment rates. This means that the researcher should 
take into account the recruitment rate preceding the selection of his sample when 
he is interpreting the data. When there have be<sn long, stable periods of steady 
recruitment, entry and termination samples will have approximately the same 
characteristics, but these stable periods have seldom occurred in the NYC. Be- 
sides the frequent changes in policies, NYC programs at the local level have been 
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in Che past, and can be expected in the future, to be deeply affected by budgetary 

freezes and thaws. 

Control Groups 

A control group is used In social research to estdjlish a baseline for 
judging program effects and should match the experimental group on all relevant 
variables except for participation in the program. Our experience suggests that 
groups which are Initially matched on age, sex, race, and school grade completed 
will also be found to be reasonably well matched on other demographic variables 
such as family badeground, recorded police contacts, and IQ test scores. In 
general, the greater the nunber of variables on which the two groups match, the 
more confident the researcher can be in the ultimate value of his results. The 
advantages and disadvantages of various sources of control group subjects are 

discussed below. 

1. R^dom Assigniiaents 

Ideally, the control group is the "same'* as the experimental group 
except that the experimental group participates in the IJYC program. SucI-. an ideal 
mi^t be approached if qualified applicants to the program could be assigned 
alternately to experimental and control groups. While no two individuals are 
exactly alike, such a method randomizes individual differences, and the two re- 
sultant groups could confidently be expected to be highly similar. As a practical 
matter, it is only possible to use this technique when there is a surplus of 
applicants over program quotas sufficiently large to comprise the control group. 
Even then, the denial of program participation to implement a research design 
raises the ethical question of denying advantages to individuals in order to 
study the effects of deprivation. 



-ERIC 

MiiiiiffiiffTigiiia 









10 - 



2. Applicants Who Are Kot Accepted for the Program 

Although it ±8 not often feasible to alternate applicants between the 
experimental and control samples, there nay be a substantial number of applicants 
who were not accepted in the program. Such individuals can be considered as can- 
didates for the control group. Since there are likely to be systematic differences 
between applicants who are accepted and those who are not, it Is important that 
the control group derived from unsuccessful applicants be matched with the experi- 
mental group on all available relevant variables. As was indicated above, in our 
experience, if we matched the groups on age, sex, race, and year of school com- 
pleted, we found that they also tended to match on variables developed from infor- 
mation later obtained in the interview process. It will be impossible, of course, 
to control for the motivational factors which may have caused the applicant to 
be accepted or not accepted in the program. It is possible that there may be 
diametrically opposite factors involved in the failure J an applicant to enroll 
in a program. On the one hand, he may be more employable and may have found a job 
prior to the time he was notified of a possible NYC enrollment. On the other hand, 
he may lack the persistence necessary to get into the NYC and may never have followed 
up on his application. The possibility that subtle motivational factors differ- 
entiate successful and unsuccessful applicants should always be kept in mind. 
Experience has shown that— notwithstanding these imponderable factors— it may be 
possible to develop useful control groups from applicant files. 

3. Employment Service Files 

The State Employoent Services, particularly the touth Opportunity Centers, 
have application, from youths who did not enter the Neighborhood Touth Corps. The 
Of these appllcsnts. If they can be made available for research purposes. 
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provide a possible source of control subjects# We have found that the most 
practical procedure is first to select individuals from the files by random 
sampling procedures tint 11 a group of a predetermined size is constituted. Sub 



stitutions are then made by random selection tii& sppXi-Cssiw populfition tsstiiiX 

the experimental and control groups are matched on sex, race, age, and School 
grade completed. 

4. Nomination from School 



An individual in the experimental group may be matched by contacting 
the school where he was formerly enrolled and attempting to locate a comparable 
individual who matches the experimental subject on all of the relevant variables 
and who is considered by the school authorities to be comparable in attitude 
and school behavior. Theoretically, this is a promising way of establishing 
control samples. But, since one Individual succeeded in enrolling in the NYC 
and the other did not, there still has been no control established for the moti- 
vational factor. Nevertheless, unless the NYC is enrolling a substantial pro- 
portion of the population, this motivational factor may not be crucial. As a 
matter of practice, however, we found that the information possessed by the 
school administrators was inadequate and the addresses were so out of date that 
the problems of locating the subjects were aggravated beyond what was customarily 



encountered. 

5. Nomination of Peers 

Subjects in the experimental sample may be asked to nominate friends 
or acquaintances who are similar to experimental subjects except that they lack 
NYC experience. We have not tried this method but feel that it offers some 
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possibilities. Of course, there would remain the motivational factors related 
to why the experimental subject was in the NYC and the other was not. On the 
other hand, the location problem might be simplified because a current address 

could usually be obtained for the control subject. 
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Data Requirements 

In evaluation research, data must be obtained relating to three types 
of variables, \diidli we have chosen to call criterion, program, and moderator 
variables. The criterion variables are the standards against which the effec- 
tiveness of the NYC program might be judged and include measures of post-RYC 
employment, community adjustment, attitude change and the like. The program 
variables are the program components or diaracterlstlcs which might influence 
the ’’success” of the program such as type of work assignment, type and quality 
of counseling, job development, and remedial education. The moderator variables 
are the characteristics of the youth which must be considered tdien studying the 
the relationship between program and criterion variables, such as family back- 
ground, employment history, sex, age, etc. Moderator variables are also used 
for matching experimental and control groups and for differential analysis of 
the data. The measurement of each of these types of data is discussed below. 
Criterion Variables 

The criterion variables, the changes in social and work adjustment 
which mi^t be attributed to NYC experience, were observed at three levels. The 
first is what happens after the youth leaves the NYC program, the second is what 
happens while the youth is in the program, and the third, observable changes 
in the youth. Measures at each of these levels were developed and are discussed 
below. 

1. Increased Employability 

Steadiness of employment, pay rate, and level of work are all considered 
ingredients of this criterion. A completely successful outcome would be steady 
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employment in a job with -an adequate pay rate offering reasonable promise of 
continued employment during the working career of the ex-enrollee. A less 
successful outcome would be steady emplo 3 nnent in a job with an adequate pay rate 
but with limited future prospects. For example, employment in unskilled work 
such as caA/ashing, even though the work is currently steady and reasonably 
well paid, might lead to future problems when the Industry is automated. A 
still poorer outcome would be intermittent employment or a low pay rate. The 
worst outcome, of course, would be no employment at all. 

The information on employmeni; is obtainable without serious problems 
through either interviews or mail questionnaires. We have not yet had any experi- 
ence getting this Information from employers, although this is planned during a 
latter phase of the research contract. 

2. Community Adjustment 

The amount of criminal or delinquent behavior of the subject is an 
important measure of bis community adjustment. One indicator of delinquent or 
criminal behavior is the number of his official contacts with the police reported 
on the police blotter. Because of the confidential nature of this information, 
it is not always obtainable. So far, we have been able to obtain it for each 
of our research sites after convincing the proper authorities of the Importance 
of the research studies and developing procedures to assure confidentiality. The 
results, however, are difficult to Interpret because of wide variation in prac- 
tices among police departments and between different socio-economic groups. We 
also found significant differences in the rate of police contacts correlated 
with age. It is essential for the Interpretation of police-contact data that 
control group comparison be available. 



- 15 - 



3. Self-Support 

The extent to which the ex-enrollee is supporting himself is another 
measurement of community adjustment. This information is obtainable through 
Interviews and mail questionnaires. A major problem in Interpreting the data 
concerns classifying the status of the married woman. A determination needs to 
be made as to the extent to \Ai±di the husband is providing adequate support for 

the family. 

4. Additional Education 

The extent to which the ex-enrollee has Improved his educational achieve- 
ments and trained himself for future jobs is an important interim measure predicting 
eventual outcomes. This Information is obtainable through questionnaires and inter- 
views but is very difficult to analyze because of the wide variations in the amount 
of time spent in these activities and the range of possible educational and train- 
ing programs. The questions need to be carefully designed and the coding of the 
responses requires the exercise of good judgment. 

5, Changes in Attitude and Behavior 

Current attitudes and behavior are Important predictors of outcomes but 

the lack of well-developed, reliable measuring instruments limits the analysis 
of these characteristics. We have done some exploratory work through use of 
Interviewer ratings of respondents. We do not know the degree of reliability of 
these measures but suspect that it is very low. We have found, however, that 
these ratings are useful for comparing large groups of individuals. Further 
work is being done in developing better instruments which can be administered 
at various stages of the NYC program. 



Progtam Variables 

The NYC program attempts to increase the employability of its etirollees 
by providing counse..-Lng, remedial education, work experience, and training in 
occupational skills. The following measurable program variables have been identified 
as possible factors influencing program outcomes whidi can be related through 
statistical analysis to the criterion variables described above. 

1. Length of work week and hourly pay. This Information is easily 

obtained. 

2. Counseling program including the qualif cations of counselors, 
approach to counseling and the setting in which it is conducted, 
and the emphasis placed on counseling. This information can be ob- 
tained from a description of the counseling program and from inter- 
views with enrollees. 

3. Characteristics of work station including type of agency, 
amount of support given to NYC program, services provide'’^ work- 
ing conditions, and attitudes of other employees toward enrollees. 

4. Nature of work experience including size of work group, skills 
required for the job, type of work, enrollee’s perception of the 
value of the job, and enrollee's interest in the job. 

5. Supervisor’s behavior including discipline, relationship with 
enrollee, attitude toward enrollee and attention paid to training. 

Moderator Variables 

The identification of relevant moderator variables is of obvious Im- 

ef f ectiveness , and the data requirements of 



portance to the analysis of program 



such variables should be carefully considered. Some important moderator vari^les 
such as sex, race, age, and school grade completed are ordinarily available at 
the time of the constitution of study groups. Other related moderator variables 
such as quality of school performance and delinquency can be developed from 
school or police records, while moderator variables such as family composition 
and welfare assistance status often can be developed from program records (for 
the experimental group) . Prior to follow-up contact, information about the ex- 
perimental group is usually more extensive than that about the control group. 

The nature of preliminary information in the experimental group may well suggest 
the types of data that should be developed from the individuals in the cciiparative 

study groups. 

In the NYC research, a more intensive analysis has suggested that en— 
rollees can be classified into types, each of which requires a somewhat different 
program emphasis. The following preliminary typology has been developed to 
guide our research and is being tested by the longitudinal data. 

1 . Disadvantaged Graduate Type 

This individual is fully socialized and has taken advantages of the 
educational opportunities available to him. He sufferes primarily from defects 
in the system and needs opportunity to acquire needed skills and to be considered 
for suitable jobs. A very limited program can often show dramatic results with 

this type. 

2. Adverse Situation Type 

Like the disadvantaged graduate, this type is reasonably well socialized 
but, because of an adverse situation such as economic need or pregnancy, dropped 
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out of school before graduation. Such aii individual primarily needs an opportunity 
but his deficiencies in education and training are f Sequent iy Sufficiently severe 
to tequire long-term remfediation. 

3. Rebel type 

An enrollee of this type has rejected society’s valufes and frequently 
has a long history of delinquent acts. He needs to develop the coiivictiori that 
it is possible to satisfy his needs through legitimate channels* and then to re- 
ceive the training which makes this possible. 

4. Low Self-Esteem Type 

An enrollee of this type has lost confidence in himself and he needs 
to develop self-esteem throu^ experiencing success. 

Each of these four types requires substantially different program em- 
phases and the same program component will affect each of them differently. The 
relationship between program and criterion variables, therefore, can be studied 
more productively by recognizing these types as moderator variables. 
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Data Collection Techniques 

There are a numiber of possible methods for collecting datai questionnaires j 
interviews, observations, official records, special reports, etc. While we ate 
using a number of sources, most of the data in our completed studies has been ob 
talned from the official records and from the subjects themselves. Most of what 
we have learned relates to the problems of collecting data from subjects and we 
are therefore limiting our discussion to this aspect of data collection. 

There are three possible ways of getting information from the subjects 
in the experimental or control groups: telephone, mail questionnaire and personal 

Interview. In our studies, primary use was made of personal interviews, and the 
extent of our experience warrants the separate consideration of interviewing in 
the next section of this report. 

In the NYC studies, the telephone was used primarily to check Information. 
Since it was felt that the quality of information obtainable through the telephone 
would not be satisfactory, the telephone was not used in data gathering. However, 
limited use was made of mailed questionnaires with the results described below. 

Mailed Questionnaires 

In one phase of our research, short questionnaires were mailed to 
former NYC enrollees and approximately one- third responded. Several months later, 
two equivilant samples of former enrollees, one from a different program in the 
same city and the other from a different city, were asked to complete question- 
naires and were promised pay of $1.50 upon receipt of the completed questionnaire. 
The response rate from both of these samples was approximately 40 percent. At 
the same time, a second mailing was sent to the initial sample with an offer to 
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pay. An additional 13 percent of the sample responded and it was concluded that 
the offer to pay and the follow-up mailing had about an equal effect on increasing 
the response rate (see Table 2). 



TABLE 2 

OUTCOMES OF MAILED REQUESTS FOR TERMINEE SELF-REPORTS, 

THREE TERMINATION SAMPLES 




Mailings and Outcomes 


Pittsburgh 

City 

(N-121) 


Pittsburg 

HWA 

(N-127) 


Cincinnati 

(N-125) 








Percent 




Completed and Returned: 

1st mailing, no remuneration 
1st mailing, remuneration 
2nd mailing, remuneration 

Total 


41 

41 


32 

13 

'45 


39 

39 


Returned by Post Office 
(not delivered) 




16 


13 


11 


No response 




44 


42 


50 


TOTAL 


101 


100 


100 



By use of a ionbination of techniques, it has proved possible to get a 
response rate of between 40 to 45 percent of former NYC enrollees through use of 
short, mailed questionnaires. The completion rate might be increased, perhaps, 
throu^ additional follow-ups and efforts to obtain current addresses. 

The differences between respondents and non-respondents in this study 
were less than might have been expected (see Table 3) . On selected variables of 
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race» sexs education, school grade completed, there were no significant differences 
between respondents and non-respondents. There was, however, a tendency for re- 
spondents to have left the program for a job or further training and for non- 
respondents to have left prematurely on their own initiative. 

TABLE 3 

COMPARISON ON SELECTED VARIABLES OF RESPONDENTS AND 
NON-RESPONDENTS, TERMINATION STUDIES^ 



Variables 



Respondents 

(N-155) 



Non-respondents 

(N-218) 



Percent 



Male, white 
Male, Negro 
Female, white 
Female, Negro 

Completed 8th grade or less 
Mean school grade completed 
In the NYC 3 months or less 

Nature of NYC termination: 
Programmed 
Administrative 
Premature: NYC initiative 

Premature: Other 

Above-average improvement in 
overall employability® 



19 


16 


35 


42 


8 


7 


38 


35 


25 


22 


9.8 


9.7 


33 


37 



49 


38 


19 


18 


13 


15 


19 


29 


46 


36 




^Pittsburgh City, Pittsburgh HWA, and Cincinnati. 
^Two Pittsburgh studies only. 
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Even under the best of circumstances, the completion rate of mailed 
questionnaires is likely to be substantially less than that of the personal inter- 
view, which has achieved a completion rate of 80 percent under optimum conditions* 
It should also be noted that a great deal more information can be obtained through 
the interview since it is probably not practical to request the respondent to 
complete more than a two-page, mailed interview. The cost of a personal interview, 
however, is much hl^er and the time span of the interview period is much longer. 
It seems thus that in many situations, the mailed questionnaire is a useful tech- 
nique for getting information about terminated enrollees and can be made more 
useful by following up non-respondents through interviews. 




IV 



Interviewing 

The characteristics of the populations being studied, different in 
sWeral important respects from populations traditionally studied in social re- 
search, tend to require interviewers with quallficatloris different from those of 

/ 

Interviewers traditionally used— white, middle-class housewives, and college stu- 
dents* The yoxmg, disadvantaged, urban Negroes who figure prominently in the 
NYC studies are very mobile, although much of their mobility is within an urban 
area, or even, an urban neighborhood. Frequently, they not only do not leave 
forwarding addresses but intentionally make location difficult. The Interviewer, 
therefore, frequently must develop locating Information from neighborhood resi- 
dents, who are often suspicious and unwilling to give Information to help locate 
siibjects. Once located, the young persons figuring in the NYC studies appeared 
to be no more difficult to interview than middle-class subjects. 

The characteristics of the population studied thus entail special 
approaches to locating subjects and conducting the interview. Our experience 
in this area is summarized in the following paragraphs. 

Qualifications of Interviewers 

In general, we have found that the usual sources of interviewers for 
survey research work do not provide the type of person needed to conduct inter- 
views with disadvantaged groups, particularly those in urban areas. The personal 
qualities described below appear to be more important than specific experience 
with any particular type of work. 
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1. Knowledge of Living Practices of Disadvantaged Groups 

It is important that the interviewer understand how disadvantaged groups 
live and know how to go ahout locating subjects. This qualification ordinarily 
rules out white interviewers, although there are exceptions. 

2. Persistence 

An important quality of an interviewer is his determination to locate 
the subject. A good interviewer takes pride in his ability to complete the inter- 
view and will use a great deal of ingenuity and persistence in locating the sub- 
ject and getting him to provide the information required for the interview. 

3. Approach 

An effective interviewer has to be skillful in getting the subject 
to agree to the interview. He has to be skilled at avoiding alarming the sub- 
ject, reassuring him, convincing him of the importance of cooperating, and 
eliciting his response until the information has been obtained. 

4. Flexibility of Work Sdiedule 

An interviewer should have hours available for interviewing which 
correspond to the times that the subjects are most likely to be home. Our ex- 
perience indicates that the best time to complete the interview for most sub- 
jects is in the early evening or on weekends. He also should have reasonably 
large blocks of time available since it has proved to be more efficient for the 
interviewer to devote a minimum of four hours at a time to the interviewing in 
order to get the best results. Since interviews often have to be conducted at 
night, or on weekends, the availability and suitability of an interviewer for 
unusual hours and inner-city work must be determined at the outset. Again, white 



o 



- 25 - 



Interviewers— particularly white female interviewers— are, prima facie, usually 
to be avoided. 

The appearance of the interviewer is an important factor determining 
the response made to him both from the point of view of locating the subject and 
achieving a completed Interview. His clothing, manner, race, and approach can 
all be important to the accomplishment of his mission. The teneclty of the inter- 
viewer and his knowledge regarding the habits of the population being surveyed 
have an ii^portant bearing on how successful he will be in locating the subject 

and securing a complete interview. 

Although, in general, the appearance of an Interviewer should be such 
as to allay the impression of being an ’’outsider," his personal qualifications 
may well enable him to succeed even though his appearance or background might 
engender suspicion. In some instances, for example, off-duty police officers 
proved to be successful interviewers, as were probation and parole officers. 

Frob^ly the best results are obtained by considering the interviewers 
as a team with different members bringing different qualities to the team effort. 
There should be a mixture of Negro and white, male and female, and different 
age groups. Interviewers should be available at different times of the day and 
days of the week so that all possible hours are covered. A team with a range 
of qualities and talents gives the supervisor flexibility in assignment and per- 
mits differential assignments so that there is the maximum likelihood that the 
interview can be completed. 

Location of Respondents 



We have found that one of the most effective time-saving techniques 
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Is to do everything possible to establish correct addresses beiEote attempting 

Interview contact with the respondent. The subjects of the NYC study samples 

proved to be highly mobile, and the addresses obtained at the time the sample 

« \ 

was selected were incorrect in about 50 percent of the cases . This moblxity is 
a function both of the age of the sample and their socio-economic status. They 
are at the age When they are leaving home and establishing tiieit oWh hdilseholds. 
The girls get married with the resultant change in names; individuals of this 
socio-economic class move frequently. For the most part, the mobility is within 
the city or even a section of the city. IThile the mobility within the city is 
much greater than it is for middle-class subjects, this population is less prone 
to make long-distance moves. It is obviotis that mobility and consequent obso- 
lescence in location information will increase if there are urban renewal or 
piiblic housing programs under construction in the research site. For all of 
these reasons, it can be expected that a significant portion of the time of the 
interviewer will be absorbed attempting to locate the subjects. 

In addition to the high mobility, it is much more difficult to get in- 
formation about where the subject may have gone than is typical for middle-class 
subjects. The subjects do not always leave forwarding addresses and, in many 
cases, may make a deliberate effort to make it difficult for bill collectors, 
police or personal enemies to locate them. Neighbors are often suspicious of 
strangers asking questions and will not supply information even when it is known. 

Because of the difficulties of locating the subjects, every possible 
method must be used systematically in order to achieve an adequate completion 
rate. The methods which we have tried will be discussed below. 
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Post Office Correction of Mailing Address 

The post office provides an inexpensive service which we have found 
win provide new addresses for about 25 percent of the subjects* Postal regula- 
tion number 123.51 provides for a procedure known as Correction of Mailing List . 
This procedure involves the preparation of a 3 X 5 index card for each person 
for whom a current address is needed. In the upper, left-hand comer of the card 
is stamped or written the name and address of the organization desiring to verify 
an address. The name and address to be verified are typed or printed clearly on 
the card or on a gummed label attached to the card leaving adequate space for 

corrections to be made. 

All the cards with addresses for one particular post office are sent 
to the postmaster of that city with a letter indicating the number of cards en- 
closed a money order to cover the charge. The minimum charge is $1.00, but 
if there are 20 cards or more, the cost is $.05 per card. For third and fourth- 
class post offices, a list of names and addresses may be submitted rather than 
separate 3X5 cards. The post office will distribute the cards to the various 
mall carriers whose responsibility it will be to check the addresses, make cor- 
rections, add the zip codes, and return the cards to the post office. When all 
of the cards have been returned, they will be forwarded to the research organi- 
zation. Occasionally, a change is made in the name (as when a girl gets married) 
or a name spelling is corrected by the mail carrier. Most important, the address 
will be marked as ”0K," corrected if information is available, or annotated as 
''addressee unknown,'* ”no such number, ‘* **moved, left no forwarding address, or 
•'no mail receptacle," etc. 
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Once the target date has been set for the survey to begin, at least 
four weeks should be allowed from the time the cards are first submitted to the 
post office until the last of them are returned. However, the time required will 
vary considerably depending on the number of cards submitted and the efficiency 
of the post office personnel. 

Another method which we have tried is the use of a registered letter. 
Since this letter must be signed for by the recipient, the researcher can obtain 

information as to whether the initial address was correct. 

We have found this to be an expensive way to locate respondents which 

does not work any better and perhaps not as well as the Correction of Mailing List 
card described above. A charge of $1.10 per addressee is made when the postman 
is requested to show to whom, when and where the letter was delivered. But, 
either the respondent or a member of his family may sign for the letter which 
means the delivery of the letter is not necessarily an assurance that the respon- 
dent *s correct address is being used. He may have moved elsewhere and the letter 
was claimed by a member of his family who intended to give it to him. 

Selective Service Forwarding of First-Class Mail 

The draft status of male subjects is public infomation and can be ob- 
tained through the local draft board. Draft registrants are required by law to 
keep their draft board Informed about their current addresses but this Information 
is considered confidential and will not be supplied by the local draft board. 
However, the draft board will forward a letter to the subject in which he can 
be requested to supply his current address. The procedures prescribed by the 




Selective Service are: 
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Any person desiring to have first-class letter forwarded 
to a registrant shall enclose that letter in an envelope showing 
no return address, with the name of addressee thereon, and bear- 
ing sufficient postage to cover mailing costs. The envelope 
containing such letter shall be mailed to the local board en- 
closed in another envelope, addressed to the registrant in care 
of the local board, together with a note, dated and signed by 
the writer of the letter, requesting that the enclosed letter 
be forwarded, stating the relationship of the writer to the 
addressee and the reason the registrant’s address is not known. 

Unless it appears to the local board that it is being used as 
as a forwarding agent for bulk mailing, the local board shall 
forward the enclosed letter to the registrant at his last 
address on record . . . 

We followed the procedure of enclosing a stamped, return 3X5 card for 
the recipient to list his name and address. In addition to forwarding the letters, 
the draft boards to whom the letters were submitted supplied information on those 
\dio were deceased or currently in the Armed Forces. This was helpful in avoiding 
unnecessary time-consuming and costly effort to locate those subjects for inter- 
viewing. 

Another aid to location efforts is to supply the field supervisor and 
interviewer with as much information about the family and friends of the respon- 
dent as possible. Subjects within the age group of the NYC enrollees frequently 
are located most easily through their parents but, to complicate matters, the 
parents often have a different name than the subject. If the names of the brothers 
or sisters and the schools they are attending are known, respondents can often 
be located by contacting the brother or sister through the school. 

The beginning point for attempting to locate the respondent should be 
the last known address. This is also true for respondents who are listed by the 
post office as having moved with no known forwarding address. If the inquiry at 
the listed address does not produce results, the interviewer should then inquire 
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around the neighborhood. The next-door neighbor, the resident who has lived in 
the area for the longest period of time, or a local store are obvious places to 
begin. These location efforts can be supplemented by the use of the following 
formal information sources: 

1. Telephone Directories — the telephone directory or the information 
operator may be a rapid source of information regarding the address 
of a respondent who has moved if the name under which the telephone 

was listed is known. 

2. Reverse or Street Address Telephone Directory — telephone companies 
have a directory which lists siibscribers by street addresses. The best 
procedure to use this list is to call the telephone company business 
office, explain the purpose of the need for information, and request 
the opportunity to check some of the names for which the interviewer 
has the street address but no house number, telephone number or the 
first name of the person in whose name the phone is listed. The re- 
verse directory usually lists streets, names of residents of each 
house number on the street and telephone numbers. 

3. City Directory — the city directory lists all residents by their 
street addresses rather than their last name, whether they have a 
telephone or not. The city directories are not usually as current 
as the street address telephone directory but are sometimes useful 
for the location of respondents who do not have a telephone. 

4, Mall Carrier — a personal contact with the mail carrier may some- 
times develop information which is more current or more complete than 
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that developed through the mail check described above. 

t i i 

5. Agency or Organizational Files— there are a number of agencies which 
might have information regarding the address of respondents such as the 
public welfare department, the social service exchange for private wel- 
fare agencies, the public housing authority, juvenile authorities and 
police files, the local Economic Opportunity Office, or the state em- 
ployment service. 

6. Specialists at Respondent Location— specialists may be used to find 
subjects who are very mobile or who are difficult to interview. When 
the easy— to— find respondents have been Interviewed, an unusually skilled 
Interviewer may be designated to track down all the difficult-to-locate 
respondents. Or, early in the interview period, a specialist may be 
hired to locate respondents. One study, for example, has employed a 
burial insurance salesman, a person familiar with the residence area, 
who is able to move about the area inquiring about respondents without 

aroxislng suspicion. 

Obtaining Interview 

It has been our experience that interviews are seldom refused directly. 
The refusal is more likely to be indirect in that the interviewee will refuse to 
identify himself, will avoid the interviewer or will say he is too busy and ask 
the Interviewer to come back on some other occasion. It is, therefore, important 
that the Interviewer establish the legitimacy of his Inquiry. It can be expected 
that strangers will be viewed with suspicion by members of a ghetto area and, 
therefore, the Interviewer must establish immediately that he has no hostile 
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Intent; he Is neither a bill collector nor a law enformement officer investigating 
a crime. Various iiitervieWers have used different techniques for avoiding siis- 
piclon. Some say it helps to identify themselves as a representative of The 
George Washington University; others say it is better to identify themselves as 
a representative of the local poverty agency. Probably, the specific strategy 
should vary depending on the situation in the neighborhood. In general, it seems 
best for the interviewer to avoid attracting attention to himself by his manner 
of dress or approach. For the most part, Negroes should be used in Negro neigh- 
borhoods and whites , in white neighborhoods . 

There are so many factors which influence the ^ility of the interviewer 
to secure an interview that it is difficult to generalize from situation to situa- 
tion. For reasons which are hard to identify, it is often possible for one inter- 
viewer to secure an interview which proved to be impossible for another. For 
these reasons, it is advisable to send out a different type of interviewer ^jheu 
the respondent refuses or appears to be evading the interview. Often by sending 
out a man Instead of a woman, or vice versa, or someone with different personal 
qualities, it is possible to secure s ich interviews. 

Conduct of the Interview 

The situations encountered are so varied that no general prescription 
can be made regarding the conduct of the interview. The interviewer must make 
the best of the situations he encounters and be prepared for almost any kind of 
distraction. Disadvantaged groups typically live in over-crowded conditions 
where privacy or a reasonable amount of quiet are likely to be almost non-existent. 

It may be impractical to attempt to move the interview to a locale where 
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prlvacy can ba obtained* The suggestion that the Interview can be conducted 
better In the Interviewer's car may be Interpreted as a sex advance or as an 
Indirect way of forcing the subject away from his home. The Interviewer, there- 
fore, must develop skill In conducting the Interview under adverse situations, 
simultaneously keeping the distractions to a minimum. One of the most difficult 
problems to handle will be that of Interference by other adult members of the 
family who Insist on participating In the Interview and supplying the answers. 
Payment for the Interview 

Another Issue which was carefully considered was the wisdom of paying 
the respondent or requesting his cooperation without pay. Our experience with 
Interviews Indicated that once the respondent could be located, it was almost 
always possible to complete the intervi.ew* On the basis of this experience and 
the advice of persons experienced with this population, it was decided not to 
pay the respondents for agreeing to be interviewed. It is possible, however, 
that payment can be justified on economic grounds to reduce the niunber of call- 
backs necessary to complete the interview. We plan to Investigate this possibility 
during future interviewing. 







